Skip to content

WIP: intent-based agency with telemetry bus and Python client#83

Merged
biosynthart merged 15 commits into
hellolifeforms:mainfrom
biosynthart:intent-based-agency
Jun 16, 2026
Merged

WIP: intent-based agency with telemetry bus and Python client#83
biosynthart merged 15 commits into
hellolifeforms:mainfrom
biosynthart:intent-based-agency

Conversation

@biosynthart

@biosynthart biosynthart commented Jun 12, 2026

Copy link
Copy Markdown
Member

Summary

Intent-based agency approach for the ecosystem simulation, decoupling client-side intent generation from the server engine core.

What this adds

Telemetry bus (server/ecosim/telemetry.py):

  • Structured event logging with JSONL file output (~/.lila/logs/<session_id>.jsonl, 50MB rotation)
  • In-memory ring buffer (5000 events) with query API (filter by tick, source, level, event, entity)
  • WebSocket broadcast to subscribers via /telemetry endpoint
  • HTTP /logs API for event queries, stats, and log download
  • Tick-packet wrapper capturing intent_emit, guard firings, spawns, removals without touching the engine core
  • Absorption logging for coherence debugging

Worker integration (server/ecosim/worker.py):

  • Per-session TelemetryBus lifecycle (start on init, flush+stop on shutdown)
  • Global telemetry registry for cross-endpoint access
  • New /telemetry WS endpoint: real-time event stream with URL-based filtering
  • New /logs HTTP API: query, stats, download
  • Absorption logging in absorb_heartbeat()
  • Recent telemetry piggybacked on tick packets for client streaming

Python client (client/python/):

  • Client-side agency logic (agency.py)
  • WebSocket protocol (websocket.py)
  • World model and reconciliation (world_model.py, reconciliation.py)
  • Log replay (replay.py)
  • pygame and imgui renderers
  • CLI entry point with argument parsing

Recent Changes

  • fa9a5bb — Removed distracting concentric ripple animation from Python client water rendering (kept radial gradient + edge ring)
  • 71d03d1 — Smooth client-side agency decoupling: positions/intents/events reconciled without snapping on tick boundaries
  • 6038b10 — Updated README
  • 02977ee — Fixed smoke test for new intent-based tick packet schema
  • f2bba3e — Moved random import into _absorb_reproduction (was unused elsewhere)
  • f83112e — Lint fixes (ruff) across telemetry, worker, and client modules
  • 37947e8 — Added telemetry bus and Python client
  • a6fb18e — Initial intent-based client agency architecture

Notes

  • Telemetry is optionally injected — zero impact on the stdlib-only core engine
  • Client agency runs independently, sending intents/positions/events to the worker via heartbeat
  • This is a work in progress; feedback on the intent abstraction and telemetry schema welcome

biosynthart and others added 15 commits June 11, 2026 12:35
Split monolithic index.html into modular JS/CSS and shift from
authoritative-server to intent-based protocol.

Server changes (engine.py, worker.py):
- Tick rate 10Hz → 0.5Hz (2s intervals)
- _build_tick_packet() emits intent fields: state, drive, motion_latent,
  ref_position, eligibility flags instead of authoritative positions
- absorb_client_positions() reconciles client-reported positions via
  soft-nudge or snap + _ack strategy
- absorb_client_events() absorbs client-reported interactions with
  validation and rate caps
- get_species_definitions() exports lightweight species reference for
  client-side agency (diet_order, flee_targets, pollination targets)
- process_request() serves arbitrary static files from viz directory

Client changes (browser/):
- Split index.html (~1200 lines) into modular structure:
  css/style.css, js/constants.js, world-model.js, agency.js,
  heartbeat.js, reconciliation.js, renderer.js, particles.js, main.js
- Agency engine evaluates drives + eligibility → target selection →
  movement execution with latent-modulated style
- Heartbeat sender reports positions/events upstream at 1Hz
- Reconciliation trusts client within bounds, nudges on divergence

Bug fixes:
- Fixed params_map → derived_params in get_species_definitions()
- Added movement_speed to species definitions for client speed init
- Fixed wander target jitter (reuse until reached)
- Added interaction cooldowns to prevent event spam
- Fixed entity position initialization from ref_position on first sight
- Cleaned up heartbeat re-init pattern (no global connect override)
- Added NaN guards in renderer

New files:
- docs/intent_based_architecture.md — full architecture doc
- server/tests/client_harness.py — headless integration test (31/31 pass)
### server/ecosim/telemetry.py (new)
Structured event logging subsystem with file writer + WebSocket broadcast:
- TelemetryBus: append-only event bus with async I/O loop, JSONL file output
  to ~/.lila/logs/<session_id>.jsonl (50MB rotation), in-memory ring buffer
  (5000 events), and WS fan-out to subscribers
- TelemetrySubscriber: WebSocket wrapper with URL-based event filtering
- Telemetry query API: filter by tick, source, level, event type, entity_id
- build_telemetry_response(): HTTP handler for /logs endpoints with stats,
  download, and filtered query support
- wrap_tick_packet(): logs intent_emit, guard firings, spawns, removals from
  tick packets without touching the engine core
- log_absorption(): captures client heartbeat absorption for coherence debugging

### server/ecosim/worker.py
Integrate telemetry throughout the simulation worker lifecycle:
- Global _telemetry_registry maps session_id -> TelemetryBus for cross-endpoint access
- SimulationSession gets optional telemetry bus injection
- log_absorption() called in absorb_heartbeat() for position/event tracking
- wrap_tick_packet() called after each step() to capture engine events; recent
  telemetry piggybacked on tick packets via _telemetry field
- New /telemetry WebSocket endpoint: real-time event stream subscriber with
  query-string filtering (firehose mode)
- New /logs HTTP API: query recent events, stats, and download log files
- Telemetry bus lifecycle: start on session init, flush+stop on session end
- Updated startup log to reflect new endpoints

### client/python/ (new)
Python client package for the lila ecosystem simulation:
- lila_client.agency: client-side intent/agency logic
- lila_client.websocket: WebSocket protocol for simulation sessions
- lila_client.world_model: client-side entity state tracking
- lila_client.reconciliation: client-server state reconciliation
- lila_client.replay: log file replay functionality
- lila_client.pygame_renderer: pygame-based rendering backend
- lila_client.imgui_view: imgui-based visualization backend
- lila_client.main: CLI entry point with argument parsing
- lila_client.constants: shared configuration constants
- pyproject.toml + uv.lock for dependency management
server/ecosim/telemetry.py:
- Remove unused  and  imports
- Replace  with  (UP045)
- Replace  with builtin  (UP041)
- Rename  to  (N806 — function-scoped const)
- Drop unused  return value from  (F841)
- Remove unused  import in

server/ecosim/worker.py:
- Rename  to  (N806 — function-scoped const)

client/python/lila_client/main.py:
- Remove unused , , and  imports

client/python/lila_client/imgui_view.py:
- Drop unused  variable in level filter checkbox loop

client/python/lila_client/replay.py:
- Remove unused  and its import

client/python/lila_client/websocket.py:
- Remove unused  import
The local import of random as _random was in absorb_client_events but
_random was only referenced inside _absorb_reproduction. Move the import
into the method where it's actually used to fix F401 (unused import) and
F821 (undefined name) lint errors.
Entity updates now use ref_position instead of position and drive
instead of state_vars. Update smoke test to read from the new fields
with fallback to old field names for backward compatibility.
- Fix dt bug in main.py: remove /1000 from time.monotonic() delta
  (time.monotonic() returns seconds, was dividing by 1000 making dt ~0.000017)

- Add continuous gravity well in agency: entities gently pulled toward
  ref_position every frame (~5% speed), preventing sudden direction changes
  when new tick targets arrive

- Per-entity reconcile queue: tick divergence targets are enqueued and
  consumed smoothly at 60fps instead of instant position snaps

- Reconcile meander: large divergence produces spiral/circle motion
  (radial pull + perpendicular wobble) rather than hard lerp snap

- Species movement_speed respects entity metadata overrides:
  songbird=4.0, butterfly=2.0 (was trait-derived ~0.3 for both)

- Redesign butterfly sprite: vertical wings + nose dot for clear
  movement direction instead of symmetric horizontal shape

- Remove instant ack snap: _ack: true no longer teleports entity
  to ref_position (server already absorbed client truth)
Each entity now has a unique sync personality (deterministic from ID hash):
- _syncPhase (0..3): staggers reaction across 4 frames so not all entities
  nudge simultaneously on each tick pulse
- _syncSpeed (0.4..1.0): varies nudge intensity per entity (sluggish vs quick)

This spreads out the reconciliation 'nudge' so the sync looks organic
rather than mechanical. Applied to both browser and Python clients.

Browser client:
- world-model.js: hashId() + sync personality in WorldEntity constructor
- reconciliation.js: staggered reconcile + speed-modulated nudge factors
- main.js: pass tick into reconcile()

Python client:
- world_model.py: _init_sync_personality() on entity creation
- reconciliation.py: staggered enqueue by phase, updated last_reconciled_tick
- agency.py: _sync_speed modulates gravity well and reconcile meander speed
- main.py: pass tick into reconcile()
Port the Python client's superior reconciliation architecture to
the browser client. Python client is the gold standard going forward.

Architecture changes:
- Replace direct-position-nudge reconciliation with queue-based
  meander: reconcile() now enqueues ref_position targets; agency
  system meanders toward them with spiral motion over ~2s
- Add continuous gravity well during normal agency (was missing)
  that gently pulls entities toward ref_position every frame
- Add smooth facing angle via spherical lerp (was snapping)
- Add per-entity sync speed modulation to gravity well

Bug fixes:
- evaluateForaging: handle bare-string diet_order entries
  (JS destructuring on strings iterates characters → crash)
- evaluateFleeing: fall through to evaluateWandering when no
  flee_targets defined (was returning bare wander object)
- evaluateMateSeeking: match Python logic — iterate all entities
  instead of relying on world.findNearestMate with distance check

Renderer:
- deer and bird now use facingAngle for smooth rotation
  instead of raw velocityX/Z (no more instant direction snaps)

Files changed:
- world-model.js: add _reconcileQueue, _reconcileIdx, facingAngle
- reconciliation.js: full rewrite — enqueue-based with helpers
- agency.js: add gravity well, meander, entityPhase, lerpAngle
- renderer.js: use facingAngle for deer/bird direction
…mity

- FLOOR_POLLINATION_RELIEF: 0.12 → 0.25 (butterflies get more hunger relief
  per flower visit, keeping average hunger low enough for reproduction)
- WATER_PROXIMITY_COLONY_FACTOR: 0.2 → 0.5 (water edges are stronger
  sanctuaries for colony health recovery, offsetting stress drain)

Together these give butterflies net negative hunger drift per pollination
cycle and let colony health stabilize near water sources instead of
monotonically draining to zero.
- POLLINATOR_POST_VISIT_COOLDOWN: 15 → 8 ticks (butterflies can revisit
  flowers faster, getting more hunger relief per unit time)
- colony_health_repro_cost_factor: 0.3 → 0.1 (reproduction costs 3× less
  colony_health, so butterflies don't drain themselves by reproducing)

Combined with previous pollination relief and water proximity boosts,
butterflies should now sustain stable populations instead of dying out.
- COLONY_STRESS_ENERGY: 0.2 → 0.35 (colony_health doesn't start draining
  until energy drops below 0.35, giving butterflies more breathing room)
- butterfly clutch_size: 2 → 3 (each reproduction spawns 3 offspring
  instead of 2, boosting population growth rate)

Completes the 4-part butterfly survival tuning: more nectar relief,
faster re-visit cooldown, cheaper reproduction cost, stronger water
sanctuary, later stress onset, and bigger clutches.
@biosynthart biosynthart merged commit 0193ba6 into hellolifeforms:main Jun 16, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant